Data Led Growth project | Pice

Expand the scope of Pice & become THE Business App

Overview

This document is to construct an experiment to see how we could increase the scope of Pice (B2B payments app) from its current avatar of allowing small businesses to pay their vendors using their credit cards. We recently launched GST payments, allowing businesses to make their GST payments via Pice in an easy and seamless manner. No app-based company has done this before at scale, we would like to change that and scale this to form a big chunk of our overall payments on the platform.

Hypothesis

If we run in-app campaigns for Business Users, then the conversion rate of business users making GST Payments will increase to 10% because of contextual messaging around the important monthly dates of GST payments and lucrative introductory discounts.

Goal

The goal of this experiment is to increase the number of business users on our platform to adopt the GST Payment feature which is a recurring and a sizeable component of a business's monthly expenditure. More businesses adopting this feature would align with Pice's goal of becoming the go-to app for businesses. Plus it is a real indicator of good business behaviour and using that as an input variable to provide suitable credit products at affordable interest rates.

Success Metrics

Worst Case Scenario: 2% Conversion rate for GST payments

Best Case Scenario: 10% Conversion rate for GST payments

Experiment Design

Context

We ran extensive call & WhatsApp communication campaigns informing our business users that they can now make GST Payment from within the pice app. The conversion rate of that campaign was less than 1%.

According to us the reason for the poor conversion was 2 fold:
1. The dates on which the campaign was run

2. The channels used for communication

In this experiment we will fix both of these and see how it impacts the conversion rates

What are we testing?

We are testing to see if the conversion rates of the number of business users making GST Payments on the platform increases.

👇🏽 This is the screen to which we will be making changes

Screenshot 2024-03-29 at 12.13.51 PM.png

Variation Design

🗓 Previous Channel & Copy of Communication

1️⃣ WhatsApp

The below message was sent via our official WhatsApp account.

2️⃣

Call - Automated IVR with a female's voice with the below script.

Namaste,
Main Pice Business Payment app se baat kar rahi hoon. Aaj GST payment karne ka aakhiri din hai, aaj hi apna GST payment Pice App mein sabse kam rate par cashback ke saath karein.
Payment karne ke liye, apna GST challan generate karein, aur Pice app mein upload karke payment poora karein.
Dhanyavad.

Previous dates chosen for the campaign were just 2 days before the last date of regular GST payments - 20th. So the campaign ran from 18-20th of the month.

⭐️ What will we be changing

The new channel of communication will be within the app. A big pop-up screen will be displayed to all business users as soon as they open the app to make their regular payments. This will happen only once every day.

The pop-ups will be shown to the user from the 10th - 25th of the month. There are 3 different main reasons to choose this date range with respect to GST Payments:

15th of the month is when businesses can start making GST payments
20th is the last date to make the GST payment for the regular type
25th is the last date to make GST payments for QRMP (Quarterly Return Monthly Payment)

👇🏽 These are the 2 variations of the pop-up graphic we will be showing users

Android Large - 2643.png Android Large - 2642.png

Audience & Sample Size

This experiment will strictly be run only on the business users of the platform.

We will choose a total of 2500 Business users, split by the following ways:

Control - 500 Business Users

These users will not be shown any kind of pop-ups when they open their apps between 10th & 25th of the month.

Variation A - 1000 Business Users

These users will be shown a pop-up that simply highlights the advantages of paying GST from Pice with no call to action. This pop-up will also contain an introductory discount on the service fee to make the GST payment.

Variation B - 1000 Business Users

These users will be shown a pop-up that highlights the advantages of paying GST along with a call to action to explore/pay now. This pop-up will also contain an introductory discount on the service fee to make the GST payment.

A/A Test

This section wouldn't be possible to do for this assignment as we haven't yet implemented an A/B testing tool in our product. However, I will make it a point that we have one in place as soon as possible now that I understand the need for conducting experiments and data-driven decisions at scale.

Brand focused courses

Great brands aren't built on clicks. They're built on trust. Craft narratives that resonate, campaigns that stand out, and brands that last.

View all courses